INTRODUCTION:

Medical equipment is defined by WHO as “Medical devices requiring calibration, maintenance, repair, user training and decommissioning – activities usually managed by clinical engineers”. Clinical engineering is the branch of biomedical engineering dealing with the actual implementation of medical equipment and technologies in hospitals or other clinical settings¹. Medical equipment is used for the specific purposes of diagnosis and treatment of disease or rehabilitation following disease or injury; it can be used either alone or in combination with any accessory, consumable or other piece of medical equipment². The global market for medical device technologies has reached nearly $521.2 billion in 2017 and is expected to reach $674.5 billion by 2022, at a compound annual growth rate (CAGR) of 5.3% for the period of 2017-2022³. The Medical Devices industry in India is presently valued at USD 5.2 billion and contributes 4-5% to the USD 96.7 billion Indian health care industry. Currently, India has about 750–800 medical device manufacturers in the country, with an average investment of Rs 170–200 million and an average turnover of Rs 450–500 million⁴. This signifies the increasing dependence of healthcare delivery on medical equipment and hence the need for efficient maintenance to ensure uninterrupted flow of services.

According to World Health Organization medical equipment maintenance is mainly divided into two categories - Inspection & Preventive Maintenance (IPM) and Corrective Maintenance (CM)⁵. Organizations perform IPM at regular intervals and CM as required. The outcome of such maintenance program can be quantified by identifying Key Performance Indicators (KPI). KPIs are the metrics or other source of information which indicate how outcomes can be measured or recognized. As stated by Armstrong⁶, performance indicator is “a statistic or other unit of information which reflects, directly or indirectly, the performance of a health or welfare intervention, facility, service or system in maintaining or increasing the well-being of its target population”. A Key Performance Indicator (KPI) helps to measure the performance of an organization against the decided goals and objectives. Once the targets for the organization are decided, it is possible to practice and maintain a record of quality of care with the help of KPIs over a period of time. KPIs link the organizational vision with the individual action. To ensure that the activities and the decided targets run parallel in an organization, there is a need to review these activities periodically. This can be obtained through performance management, setting the desired targets and then measuring those activities. Such performance systems should be both quantitative and qualitative⁷. Measuring the performance of any management system is an attempt to achieve the long term goals and objectives of any organization. These efforts in turn can be translated in terms of organizational success⁸. This can be facilitated by benchmarking KPIs, thus bringing improvements in quality and safety over time.

Benchmarking when applied to organizations (eg. institutions, companies etc) makes comparison with other organizations which helps to find gaps and adapt their specific best practices, with the aim of increasing performance⁹ Benchmarking is a measure of the quality of organization’s policies, products, programs, strategies etc. and their comparison with standard measurements or similar measurements of its peers. Benchmarking is a way of discovering what is the best performance being achieved – within an organization, by a competitor or by an entirely different industry. This information can then be used to identify gaps in an organization's processes in order to achieve a competitive advantage¹⁰. It helps to identify various opportunities for improvement and effectively monitor the changes once the improvements have already occurred¹¹.

MATERIALS AND METHODS:

A Retro-prospective mix method study (Qualitative study– stakeholder interviews; Quantitative study – Indicator performance) was conducted with an aim to evaluate the medical equipment maintenance management in a tertiary care teaching hospital. The study had following objectives:

1 To identify the key performance indicators which reflect the performance level of the maintenance service of the biomedical engineering department.

2 To set benchmarks for all the identified KPIs

3 To evaluate the medical equipment maintenance management after the benchmarking activity

Nine in-depth interviews were conducted for the stakeholders in medical equipment maintenance management in the hospital i.e., employees from Biomedical Engineering, Intensive Care Unit, Radiology and Quality department. An interview guide was used as tool. The questions in the interview guide were framed based on literature review. The questions covered different aspects of key performance indicators such as financial aspect, technical aspect, organisational aspect and risk management. An audio recording of the responses was taken with the informed consent of the participants. The responses were then transcribed and analyzed using Atlas Ti 8 software.

The following indicators were identified from the interviews

a. Cost to Service ratio

b. Average Repair Time

c. Uptime

d. Age Failure Rate

e. Planned Preventive maintenance Completion Rate

f. Planned Preventive Maintenance Yield

g. Delinquent Work Order

h. Customer Service

i. Number of incident reports per month

j. Uptime for life saving equipment

k. Percentage Lifesaving equipment downtime to total downtime

After determining KPIs, the relevant data from the months of October 2016 to March 2018 was collected from the records maintained by the hospital biomedical engineering department. Existing service standards were used as benchmarks for indicators where available. For evaluation of KPIs with no existing service standards, internal benchmarking process was used as a tool. For creating internal benchmarks, data collected from October 2016 to September 2017 was taken and the 75^th percentile value was chosen as the benchmark for positive indicators while 25^th percentile value was chosen as the benchmark for negative indicators.

The values of the KPIs obtained from the data collected between October 2017 to March 2018 were compared against the benchmarks to evaluate the medical equipment maintenance management. Post evaluation, the reasons for discrepancy between the measured indicators and benchmarks were identified by conducting an informal interview with the manager of the biomedical engineering department.

RESULTS:

The results are presented for each KPI.

a. Cost to Service ratio:

The cost to service ratio is a useful measure in determining the financial effectiveness of a maintenance program.

Cost to service ratio = (Annual maintenance cost of the equipment/Initial cost of medical equipment) * 100

Benchmark Value: 5% to 10%

Total Cost of all Equipment		Rs 46,16,37,216
Service Cost Of all Equipment		Rs 2,30,81,861

Cost to service ratio = (46,16,37216/2,30,81,861)*100 = 5%

Cost to service ratio is 5% which is within the benchmark value. Thus, the financial performance of the biomedical engineering department can be considered as satisfactory.

b. Average Repair time:

Repair time is the time elapsed between the response to breakdown to completion of repair of the said breakdown.

Average repair time was calculated as the median of the difference between the response time and the completion time of all breakdowns occurring in one particular month.

Benchmark was determined using the internal benchmarking process as there were no pre-existing service standards.

Benchmark Value: 2.52 days

Figure 1: Average repair time against benchmark

c. Uptime:

Uptime is defined as the time during which the equipment is in operation.

Uptime = ((Available time – Breakdown time)/Available time) * 100

Benchmark was calculated by internal benchmarking using retrospective data.

Benchmark Value: 99.275%

Figure 2: Uptime against Benchmark

d. Age Failure Rate

Age failure rate is calculated as the time during which the equipment was unavailable for use based on the age of the equipment. The age failure rate was calculated for all equipment belonging to the following categories, i.e., equipment less than 3 years of age, equipment of 3-8 years, equipment greater than 8 years of age.

The existing service standard set by the biomedical engineering department was chosen as the benchmark.

Benchmark Values for different categories: < 3 years = 5%

3-8 years = 10%

>8 years = 15%

Figure 3: Age failure rate of equipment < 3 years against benchmark

Figure 4: Age failure rate of equipment 3-8 years against benchmark

Figure 5: Age failure rate of equipment > 8 years against benchmark

The age failure rate of all equipment was well below the benchmark which indicated good performance as per service standards. The failure rate was low even for equipment above 8 years of age

e. Planned Preventive maintenance (PPM) Completion Rate:

The planned preventive maintenance completion rate is percentage of procedures completed. It can be measured at the end of an assignment period.

PPM Completion Rate = (PPM completed/PPM scheduled) *100

Benchmark = 100%, as per service standards

Figure 6: PPM completion rate against benchmark

Root cause analysis revealed that failure to reach the benchmark can be accredited to,

· Problems identified during PPM of one equipment had to be followed up due to which the other equipment PPM was delayed

· Failure in quickly locating portable equipment caused delay

· Attrition among the biomedical engineering staff

f. Planned Preventive Maintenance Yield:

PPM Yield = (No. of work orders where problems were identified/PPM scheduled) * 100

Benchmark = 2.09%, calculated using retrospective data from Oct-16 to Sep-17

Figure 7: PPM yield against benchmark

PPM yield reached the benchmark in the month of October, November and January. It was particularly high in the months of February and March indicating a decline in performance.

Root cause analysis revealed following factors:

· Problems were not reported by users and work orders were sent after PPM was done.

· Failure in locating portable equipment for a prolonged period of time led to increase in work orders for such equipment 3

g. Delinquent Work Order:

Delinquent work orders are work orders not completed within 30 days.

Percentage delinquent WO = (No. of delinquent WO/Total no. of work orders) * 100

Benchmark = 2.65%, calculated as per internal benchmarking process

Figure 8: Percentage delinquent WO against benchmark

The percentage delinquent WO to total number of WO did not reach the benchmark during the course of the study indicating a drop in the performance when compared to the retrospective data which was used to arrive at the benchmarking value.

The increase in number of delinquent work orders was be attributed to

· Delay in procurement of spares due to imports

· Delay due to negotiation with the vendors

· Delay in payment of Annual Maintenance Contract (AMC) /Comprehensive Annual Maintenance Contract amount (CAMC)

· Unavailability of spares of older equipment

· Delay in repair from the vendors’ side

h. Customer Service:

Customer service was assessed based on following factors

· Accessibility of engineers

· Response to Breakdown Maintenance

· Response to Emergency Maintenance

· Quality of repair

· Quality of PPM

· Feedback on equipment status

· Courteousness of service personnel

Figure 9: Customer feedback against benchmark

· Feedback forms were collected from 119 user departments from the hospital and the score given was well above the benchmark which indicated that the user departments were immensely satisfied with the services provided by biomedical engineering department. The lowest score given was 80% which was still above the set benchmark.

i. Number of incident reports per month

Figure 10: Department-wise distribution of incident report

The number of incidents reported were above the stated benchmark except in the months of October and March. Highest number of incidents were recorded in trauma triage and NICU departments.

The rationale behind the rise in reported incidents was credited to

· Improvement in reporting which can be considered as a positive development.

· Lack of training among the user department. While the biomedical engineering department schedules user training periodically, all the users are unable to attend due to their busy schedule.

· Mishandling of equipment

· Restless patient

· High user staff attrition which implies that newer staff would be relatively less experienced in handling of the equipment and may not obtain the necessary training.

· Lack of accountability due to multiple users.

j. Uptime for life saving equipment:

Uptime is defined as the time during which the equipment is in operation.

Uptime = ((Available time – Breakdown time)/Available time) * 100

Benchmark for the uptime of life saving equipment was calculated by internal benchmarking using retrospective data.

Benchmark = 98.19%

Figure 11: Life saving equipment Uptime percentage

The benchmark for uptime of life saving equipment was not met except in the month of December.

The reason behind the decline in performance was attributed to the following factors

· Age of the equipment. In 2010, a large number of equipment was purchased by the hospital due to commissioning of new hospital block which included critical care areas. Since most equipment in these areas are now old, they are subjected to frequent breakdown which increases the downtime statistic.

· The equipment with most frequent breakdown is the haemodialysis unit which is classified under lifesaving equipment.

· Difficulty in obtaining spare parts

k. Percentage Lifesaving equipment downtime to total downtime

Percentage lifesaving equipment downtime to total downtime = (Lifesaving equipment downtime / Total downtime) * 100

Benchmark = 27.58%

Figure 12: Percentage lifesaving equipment downtime to total downtime.

· The downtime of life saving equipment did not satisfy the prescribed standards as downtime was well above the benchmark except in the month of January.

· The reason for dip in the percentage of lifesaving equipment downtime to total downtime in the month of January was due to unusual surge in the breakdown of other equipment.

· Higher percentage of downtime of life saving equipment compared to other equipment can be mainly attributed to the age of the equipment.

DISCUSSION:

The study attempted to evaluate the performance level of the maintenance management program of biomedical engineering department of the hospital which is outsourced. The analysis determined if the services provide were up to the standards and it was found that all key performance indicators compared to the service standards were above the benchmarks set by the outsourcing organization. This indicated satisfactory service from the outsourced agency. However, Miguel-Cruz et al.,¹² reported that outsourcing should be reconsidered, particularly in a government setting. Our study was conducted in a private hospital. In a study¹³ conducted to apply six sigma to reduce the downtime of medical equipment it was found that the highest percentages of breakdown lie within equipment of Diagnostic and Life support which is similar to the findings of this study where the breakdown of the lifesaving equipment was relatively higher when compared to other categories of equipment. The analysis of the Key Performance Indicators with pre-existing service standards indicated the performance was well above standards. However, the analysis of indicators evaluated using internal benchmarks showed that the benchmarks were not met in most cases.

Our study revealed that the training programs were not attended by users owing to their busy schedule. Inappropriate handling of equipment due to inadequate training of users led to unnecessary repairs and affected patient outcomes, particularly for critical equipment such as ventilators. The importance of understanding ventilator settings and modes in care delivery is emphasized in the paper titled “Top 10 Care Essentials for Ventilator Patients” by

Nirmala Jyothi¹⁴. In the study conducted by Raju P. et al.¹⁵, on the knowledge of nurses regarding mechanical ventilation, one aspect of the questionnaire assessed knowledge on mechanical ventilator and identified that only experienced nurses handled equipment appropriately. The study identified that structed training program on equipment handling for all users can improve patient outcome and thus, substantially reduce equipment downtime. In another study conducted by Chinna Devi M. ¹⁶ on Mechanical Ventilator use in Neonatal ICU (NICU), the need for nursing staff to keep pace with technological advances and incorporation of equipment maintenance information in nursing training programs has been emphasized. The researcher opined that the technological advances demand more qualified and specially trained nurses, particularly in NICU’s.

CONCLUSION:

The study has an organizational significance as the biomedical engineering department is outsourced and it is imperative for the organization to assess the performance of the services provided to ensure organizational efficiency and quality care to the patient. Delinquent work orders were identified as the most important indicator which directly impacts the organizational efficiency. By employing benchmarking activity, the vendors with frequent delinquent work orders can be identified and procurement from such vendors can be avoided in the future. A simplified operating manual with common troubleshooting problems must be made available to user departments to avoid raising of trivial complaints. This ensures efficient utilization of biomedical equipment maintenance personnel’s time in addressing complex issues which requires expertise. Such operating manuals can also be used to train junior staff and students. This is particularly helpful to the night duty staff where the engineer is available only on call. A centralized supply of lifesaving equipment such as ventilators, infusion pumps, cardiac monitors should be made available in critical care areas to ensure that there is no hindrance to patient care due to shortage of equipment

REFERENCES:

1. Mohd. Yaqub Khan, Poonam Gupta, Vikas Kumar Verma. A Review- Biomedical Engineering-Present and Future Prospective Asian J. Pharm. Res. 3 (4); 2013: 202-206

2. Available from URL: https://www.who.int/medical_devices/definitions/en/

3. Available from URL: https://www.bccresearch.com/market-research/healthcare/medical-devices-technologies-and-global-markets-hlc170c.html

4. Available from URL: http://www.makeinindia.com/article/-/v/sector-survey-medical-devices

5. Medical equipment maintenance programme overview, WHO Medical device technical series 2011. Available from URL: http://www.whqlibdoc.who.int/publications/2011/9789241501538_eng.p df.

6. Armstrong M. Armstrong's Handbook of Performance Management - An Evidence-Based Guide to Delivering High Performance. Kogan Page Publishers, London. 2014.

7. C. Stenstorm and D. Galar. Maintenance Audit Based on Quantitative and Qualitative Elements for Health Care Systems. Available at URL: https://www.diva-portal.org/smash/get/diva2:979196/FULLTEXT01.pdf

8. Poonam Chaudhary, Mahavir Singh, Vipin Kaushal, Raman Sharma, Puneeta Ajmera. Development and Validation of a Measuring Tool for Medical Equipment Management System. International Journal of Innovative Research in Science, Engineering and Technology. 5(6); 2016: 9613 - 8620

9. Ms. Ponnezhil P., Mr. Mohamad Mohayaddin A. Asian J. Nursing Edu. and Research 2(4); 2012 Available at http://www.anvpublication.org/ajner.htm;

10. J Delayne Stroud. Understanding the Purpose and Use of Benchmarking. Available at URL: https://www.isixsigma.com/methodology/benchmarking/understanding-purpose-and-use-benchmarking/

11. Patrick Kitcher. Benchmarking Performance Improvement Indicators for the Clinical Engineering Department. Available at URL : https://ieeexplore.ieee.org/document/1629790?reload=true&arnumber=1629790&punumber%3D10848=

12. Dr Apurva Tiwari, Ankita Tiwari. Performance Evaluation of Outsourced Medical Equipment Maintenance Service in a Tertiary Care Hospital. International Journal of Scientific and Research Publications. 4(9); 2014: 1-6

13. Miguel-Cruz A, Rios-Rincón A, Haugan GL. Outsourcing versus inhouse maintenance of medical devices:

1. a longitudinal, empirical study. Revista Panamericana de Salud Publica. 35(3); 2014: 193-199.

14. Nirmala Jyothi. Top 10 Care Essentials for Ventilator Patients. Asian J. Nursing Edu. and Research 2(4): 2012

15. Mr. P. Raiju, Mr. Gireesh G.R, Mr. Sachina B.T, Ms. Ann Barnes; Effectiveness of Structured Teaching Programme on Knowledge regarding modes of Mechanical Ventilator among Staff Nurses at a selected Hospital, Bangalore. Asian J. Nursing Edu. and Research 5(1): 2015

16. Chinna Devi M. Effectiveness of a Planned Teaching Programme on Care of Babies on Mechanical Ventilator among Staff Nurses working in ICU in Selected Hospitals at Bangalore. International Journal of Advances in Nursing Management 3(3): 2015

Received on 25.08.2018 Modified on 10.10.2018

Research J. Pharm. and Tech 2019; 12(1): 202-208.

DOI: 10.5958/0974-360X.2019.00037.4